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EXECUTIVE  SUMMARY 

Weston  Solutions,  Inc.  (WESTON®)  performed  an  Environmental  Security  Technology 
Certification  Program  (ESTCP)  Live  Site  Demonstration  using  a  Time-domain  Electromagnetic 
Multi-sensor  Towed  Array  Detection  System  (TEMTADS  2x2)  at  the  North  Ramp  Parking  (NRP) 
area,  Andersen  Air  Force  Base,  Guam.  The  demonstration  was  integrated  with  an  ongoing 
munitions  and  explosives  of  concern  (MEC)  removal  action  project  being  performed  in  advance 
of  military  construction  (MILCON)  activities.  Dynamic  detection  surveys  using  TEMTADS  2x2 
were  perfonned  across  2.97  acres  of  the  NRP  area  following  a  traditional  EM61-MK2  survey.  A 
total  of  970  anomalies  were  selected  from  the  TEMTADS  2x2  dynamic  survey  data.  An  additional 
225  anomalies  were  selected  from  EM61-MK2  data  that  did  not  overlap  with  the  TEMTADS  2x2 
survey.  Each  of  the  1,195  anomalies  were  reacquired  and  interrogated  using  cued  data  collection 
with  the  TEMTADS  2x2.  All  targets  of  interest  (TOI)  were  correctly  identified  during  the 
demonstration.  The  classification  process  resulted  in  correctly  identifying  100%  of  targets  of 
interest  (TOI)  and  reduced  the  number  of  clutter  or  non-munitions  related  material  that  would 
require  investigation  by  81%. 
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1.  INTRODUCTION 


This  is  one  of  a  series  of  Environmental  Security  Technology  Certification  Program  (ESTCP) 
demonstrations  of  classification  technologies  for  munitions  response.  This  demonstration  is 
designed  to  evaluate  the  classification  methodology  at  a  site  with  a  diversity  of  munitions  types 
(20  millimeter  (mm)  to  100-lb  bombs).  The  Time-domain  Electromagnetic  Multi-sensor  Towed 
Array  Detection  System  (TEMTADS  2x2)  was  demonstrated  at  Andersen  Air  Force  Base  (AFB), 
Guam  in  both  dynamic  and  cued  mode  by  Weston  Solutions,  Inc.  (WESTON®). 

1.1  BACKGROUND 

The  Military  Munitions  Response  Program  (MMRP)  is  constrained  by  available  resources. 
Remediation  of  the  entire  inventory  using  current  practices  is  cost  prohibitive  within  current  and 
anticipated  funding  levels.  With  current  planning,  estimated  completion  dates  for  munitions 
response  on  many  sites  are  decades  away.  The  Defense  Science  Board  (DSB)  observed  in  its  2003 
report  that  significant  cost  savings  could  be  realized  if  successful  classification  and  differentiation 
between  munitions  and  other  sources  of  anomalies  could  be  implemented.  If  these  savings  were 
realized,  the  limited  resources  of  the  MMRP  could  be  used  to  accelerate  the  remediation  of 
Munitions  Response  Sites  (MRSs)  that  are  currently  forecast  to  be  untouched  for  decades. 

To  build  on  the  success  of  previous  studies  and  address  resource  issues,  ESTCP  funded  a 
demonstration  of  the  TEMTADS  2x2  instrument  at  Andersen  AFB.  This  site  was  selected  as  a 
demonstration  site  because  it  contains  a  wide  range  of  World  War  II  (WWII)-related  targets  of 
interest  (TOIs).  The  site  is  also  undergoing  a  munitions  and  explosives  of  concern  (MEC)  removal 
action  in  advance  of  military  construction  (MILCON)  activities. 

1.2  OBJECTIVE  OF  THE  DEMONSTRATION 

The  overall  objective  of  the  demonstration  was  to  validate  classification  technology  at  the  North 
Ramp  Parking  (NRP)  area  at  Andersen  AFB.  The  NRP  area  was  undergoing  an  active  MEC 
removal  action  in  advance  of  MILCON  activities.  The  advanced  geophysical  classification 
activities  were  integrated  with  the  ongoing  removal  action.  WESTON  perfonned  the  following 
tasks  to  achieve  this  overall  objective: 

•  Installed  an  instrument  verification  strip  (IVS)  adjacent  to  the  NRP  area. 

•  Perfonn  a  dynamic  detection  survey  using  NRL  TEMTADS  2x2  in  a  subset  of  the  NRP 
area. 

•  Perfonn  static,  cued  target  interrogation  using  the  TEMTADS  2x2  on  anomalies  selected 
from  both  TEMTADS  2x2  and  EM61-MK2  datasets. 

•  Processed  cued  geophysical  data  to  correctly  classify  TOI. 
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1.3  REGULATORY  DRIVERS 

The  MMRP  is  charged  with  characterizing  and,  where  necessary,  remediating  MRSs.  When  an 
MRS  is  remediated,  it  is  typically  mapped  with  a  geophysical  system,  based  on  either  a 
magnetometer  or  an  electromagnetic  induction  (EMI)  sensor,  and  the  locations  of  all  detectable 
signals  are  excavated.  Many  of  these  detections  do  not  correspond  to  munitions,  but  rather  to 
hannless  metallic  objects  or  geologic  features.  Field  experience  indicates  that  often  in  excess  of 
90%  of  objects  excavated  during  the  course  of  a  munitions  response  are  found  to  be  nonhazardous 
items.  Current  geophysical  technology,  as  it  is  traditionally  implemented,  does  not  provide  a 
physics-based,  quantitative,  validated  means  to  discriminate  between  hazardous  munitions  and 
nonhazardous  items. 

With  no  information  to  suggest  the  origin  of  the  signals,  all  anomalies  are  currently  treated  as 
though  they  are  intact  munitions  when  they  are  dug.  They  are  carefully  excavated  by  unexploded 
ordnance  (UXO)  technicians  using  a  process  that  often  requires  expensive  safety  measures,  such 
as  barriers  or  exclusion  zones.  As  a  result,  most  of  the  costs  to  remediate  an  MRS  are  currently 
spent  on  excavating  targets  that  pose  no  threat.  If  these  items  could  be  detennined  with  high 
confidence  to  be  nonhazardous,  some  of  these  expensive  measures  could  be  eliminated  or  the  items 
could  be  left  unexcavated  entirely. 
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2.  TECHNOLOGY 


This  demonstration  consisted  of  dynamic  and  cued  data  collection  with  the  TEMTADS  2x2 
advanced  geophysical  sensor  system.  Analysis  of  the  data  were  perfonned  using  conventional  and 
advanced  data  processing  methods  to  select  anomalies  from  the  advanced  sensor  dynamic 
detection  data,  and  then  extract  features  and  perfonn  anomaly  classification  on  the  advanced 
sensor  cued  data. 

2.1  TECHNOLOGY  DESCRIPTION 

2.1.1  TEMTADS  2x2 

The  TEMTADS  2x2  is  an  adaptation  of  the  Naval  Research  Lab’s  standard  TEMTADS  5x5 
element  sensor  configuration  using  a  smaller  2x2  element  array.  The  TEMTADS  2x2  consists  of 
four  35cm  transmit  coils  with  four  8cm  tri-  axial  receiver  cubes.  The  receiver  cubes  are  similar  in 
design  to  those  used  in  the  second-generation  Advanced  Ordnance  Locator  and  the  Geometries 
MetalMapper)  system  with  dimensions  of  8  cm  rather  than  10  cm.  It  is  as  reliable  as  the  original 
TEMTADS,  but  its  portability  and  smaller  size  enables  access  to  difficult  terrain  where  mobility 
is  limited.  The  center-to-center  distance  between  the  transmit  coils  is  40  cm  yielding  an  80  cm  x 
80  cm  array.  The  array  is  deployed  on  a  set  of  wheels  resulting  in  a  sensor  height  of  approximately 
18  cm. 

The  transmitter  electronics  and  the  data  acquisition  computer  are  mounted  on  the  operator 
backpack,  and  a  GPS  antenna  and  an  inertial  measurement  unit  (IMU)  are  mounted  above  the 
center  of  the  TEMTADS  2x2  sensor  array.  The  TEMTADS  2x2  can  be  operated  in  two  modes; 
dynamic,  or  detection  mode,  and  cued  mode.  Data  collection  is  controlled  in  dynamic  mode  using 
the  EM3DAcquire  application  suite,  similar  to  that  used  for  the  Geometries  MetalMapper  systems. 
Custom  software  written  by  NRL  is  used  for  cued  data  acquisition.  In  cued  mode,  the  locations  of 
previously-identified  anomalies  are  reacquired  and  flagged  prior  to  being  cued  with  the 
TEMTADS  2x2. 
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Figure  2-1.  TEMTADS  2x2 


2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  major  advantage  of  the  advanced  EMI  sensor  and  the  UX- Analyze  software  is  that  used 
together  they  provide  the  ability  to  classify  anomalies  as  being  either  TOI  or  non-TOI. 
Conventional  DGM  sensors  (e.g.,  EM61-MK2)  have  very  limited  ability  to  discriminate  between 
TOI  and  non-TOI.  Other  advanced  EMI  sensors  (e.g.,  Geometries  MetalMapper,  Berkeley  UXO 
Discriminator,  Man-Portable  Vector)  have  also  been  successful  in  Strategic  Environmental 
Research  and  Development  Program  (SERDP)  and  ESTCP-funded  classification  demonstrations; 
however,  they  were  not  used  during  this  live  site  demonstration. 

The  TEMTADS  2x2  is  a  man-portable  sensor,  however,  the  associated  components  (tablet,  heavy 
backpack  and  batteries)  limit  the  ability  of  an  operator  to  collect  data  without  assistance  from 
additional  personnel.  For  this  demonstration  two  personnel  were  utilized  during  the  data  collection; 
one  person  to  operate/navigate  the  sensor,  and  a  second  to  operate  the  tablet  and  place  navigational 
aids.  The  TEMTADS  2x2  sensor  is  also  not  ruggedized  to  withstand  inclement  weather  (light  rain, 
snow,  etc.).  This  proved  problematic  during  this  demonstration,  as  the  climate  on  Guam  is  hot, 
humid,  with  frequent  precipitation. 
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3.  PERFORMANCE  OBJECTIVES 

The  performance  objectives  for  this  demonstration  are  summarized  in  Table  3-1  and  Table  3-2. 

Table  3-1  lists  the  performance  objectives  for  all  field  activities.  These  apply  to  all  detection  and 
classification  work  performed  in  the  NRP  area.  Table  3-2  lists  the  perfonnance  objectives  for  the 
advanced  classification  activities.  These  objectives  apply  to  all  similar  work  perfonned  using  NRP 
area  advanced  classification  data. 


Table  3-1.  Performance  Objectives  for  Field  Activities 


Performance  Objective 

Metric 

Success  Criteria 

Adv.  Sensors  Survey: 

Down-track  location  ±25  cm 

Repeatability  of  Instrument 
Verification  Strip  (IVS) 
measurements 

Amplitude  of  EM  anomaly 

Measured  target  locations 

Adv.  Sensors  Cued: 

Library  match  >90%  using  3- 
criterion  metric  with  equal 
weighting  to  the  three  criteria  using 
first  day’s  IVS  inversion  as  the 
library  item. 

Complete  coverage  of  the 
demonstration  site 

Footprint  coverage  calculated  using 
UX-Process  Footprint  Coverage 
Quality  Control  (QC)  tool;  excludes 
inaccessible  areas. 

>=  85%  coverage  at  0.50-m  line 
spacing;  and 

>=  98%  coverage  at  0.60-m  line 
spacing 

Along-line  measurement  spacing 

Point-to-point  spacing  from  data  set 

98%  <  25-cm  along-line  spacing 

Detection  of  all  TOI 

Percent  detected  of  TOI 

100%  of  TOI  detected  within  40- 
cm  halo  of  the  surveyed  location 

Cued  interrogation  of  anomalies 

Instrument  position 

100%  of  anomalies  where  the 
center  of  the  instrument  is 
positioned  within  40  cm  of  actual 
target  location 
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Table  3-2.  Performance  Objectives  for  Advanced  Classification  Activities 


Performance  Objective 

Items 

Metric 

Success  Criteria 

Correctly  classify  QC 
seeds  and  correctly 
classify  native  and 
population  seed  items 

All  seeds  and  all  native 

TOI 

Percent  classified  as  TOI 

100%  classified  as  TOI 

Correctly  identify  group 

All  TOI  and  all  excavated 
non-TOI 

Percent  of  TOI  and 
excavated  non-TOI 
grouped  correctly 

85%  correctly  grouped  in 
the  small,  medium,  and 
large  groups 

Correct  estimation  of 
extrinsic  target 
parameters 

All  excavated  anomalies 

Measured  location  and 
depth  to  center  of  mass  of 
recovered  items 

X,  Y  <15  cm  ( 1  (7  ) 

Z  <  10  cm  (1  O' ) 

Maximize  correct 
classification  of  non-TOI 

All  non-TOI 

Number  of  false  alarms 
eliminated 

Reduction  of  clutter  digs 
by  >50%  while  meeting 
all  other  demonstration 
objectives 

Minimize  number  of 
anomalies  that  cannot  be 
analyzed 

All  cued  anomalies 

Number  of  anomalies  that 
must  be  classified  as 
“Unable  to  Analyze” 

Reliable  target  parameters 
can  be  estimated  for  > 

95%  of  anomalies  on  each 
sensor  anomaly  list. 

3.1  OBJECTIVE:  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  STRIP 
MEASUREMENTS 

The  reliability  of  the  survey  data  depends  on  the  proper  functioning  of  the  survey  equipment.  This 
objective  concerns  the  twice-daily  confirmation  of  sensor  system  perfonnance. 

3.1.1  Metric 

The  metrics  for  this  objective  are  the  down-track  position  of  the  maxima  for  the  TEMTADS  2x2 
when  used  in  dynamic  survey  mode,  and  the  percent  match  of  the  inverted  data  to  the  library  for 
the  specific  seed  items  when  surveying  in  cued  mode.  These  metrics  are  applied  for  each  of  the 
twice-daily  surveys  of  the  IVS. 

3.1.2  Data  Requirements 

The  IVS  data  were  used  to  judge  this  objective.  For  cued  surveys,  the  first  day’s  IVS  measurement 
over  each  Industry  Standard  Object  (ISO)  was  used  as  the  library  basis  for  all  future  IVS 
comparisons  during  the  project. 

3.1.3  Success  Criteria 

This  objective  is  considered  a  success  for  dynamic  survey  data  if  the  down-track  position  of  the 
anomaly  is  within  25  cm  of  the  seed  item’s  known  location.  The  objective  is  considered  to  be  met 
in  cued  mode  if  the  library  matches  are  equal  to  or  greater  than  90%. 
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3.2  OBJECTIVE:  COMPLETE  COVERAGE  OF  THE  DEMONSTRATION  SITE 


The  reliability  of  the  survey  data  depends  on  the  extent  of  coverage  of  the  site.  This  objective 
concerns  the  ability  of  WESTON  to  completely  survey  the  site  and  obtain  valid  TEMTADS  2x2. 

3.2.1  Metric 

The  metric  for  this  objective  is  the  footprint  coverage  as  measured  by  the  UX-Process  Footprint 
Coverage  QC  tool. 

3.2.2  Data  Requirements 

A  mapped  data  fde  was  used  to  judge  the  success  of  this  objective. 

3.2.3  Success  Criteria 

This  objective  is  considered  a  success  if  the  TEMTADS  2x2  dynamic  survey  achieved  at  least 
85%  coverage  at  0.5-m  line  spacing  and  98%  at  0.6-m  line  spacing  calculated  using  the  UX- 
Process  Footprint  Coverage  QC  tool. 

3.3  OBJECTIVE:  ALONG-LINE  MEASUREMENT  SPACING 

The  reliability  of  the  survey  data  depends  on  the  measurement  density.  This  objective  concerns  the 
ability  of  WESTON  to  acquire  sufficiently  dense  measurements  to  obtain  valid  data. 

3.3.1  Metric 

The  metric  for  this  objective  is  the  point-to-point  distance  as  measured  using  UX-Process  point- 
to-point  distance  tool. 

3.3.2  Data  Requirements 

A  mapped  data  file  was  used  to  judge  the  success  of  this  objective. 

3.3.3  Success  Criteria 

This  objective  is  considered  a  success  for  dynamic  TEMTADS  2x2  surveys  if  98%  of  the  data 
have  along-line  spacing  of  25  cm  or  less. 

3.4  OBJECTIVE:  DETECTION  OF  ALL  TOI 

Quality  data  should  lead  to  a  high  probability  of  detecting  the  TOI  at  the  site.  This  metric  applies 
only  to  the  detection  phases  of  work  and  is  specific  to  those  items  defined  as  detectable,  which  for 
this  project  is  initially  defined  as  peak  signal  7  times  site  root  mean  square  (RMS). 

3.4.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  seed  items  that  are  detected  using  the  specified 
anomaly  selection  threshold. 
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3.4.2  Data  Requirements 

A  target  list  was  generated  by  WESTON  and  compared  against  the  surveyed  locations  of  the  seed 
items. 

3.4.3  Success  Criteria 

The  objective  is  considered  a  success  if  100%  of  the  seeded  items  are  detected  within  a  40-cm  halo 
of  their  surveyed  locations. 

3.5  OBJECTIVE:  CUED  INTERROGATION  OF  ANOMALIES 

The  reliability  of  cued  data  depends  on  acceptable  instrument  positioning  during  data  collection  in 
relation  to  the  actual  anomaly  location. 

3.5.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  anomalies  that  are  within  the  acceptable  distance 
from  the  center  of  the  instrument  to  the  actual  target  location  during  data  collection. 

3.5.2  Data  Requirements 

The  location  of  the  center  of  the  TEMTADS  2x2  array  at  each  cued  anomaly  was  compared  against 
the  measured  locations  of  items  recovered  during  intrusive  investigations. 

3.5.3  Success  Criteria 

The  objective  is  considered  a  success  if  the  center  of  the  instrument  is  positioned  within  40  cm  of 
the  actual  anomaly  location  for  100%  of  the  cued  anomalies. 

3.6  OBJECTIVE:  CORRECTLY  CLASSIFY  QC  SEEDS  AND  CORRECTLY 
CLASSIFY  NATIVE  AND  POPULATION  SEED  ITEMS 

This  metric  applies  to  QC  seeds,  population  seeds,  and  native  TOI.  Seed  items  are  used  to  provide 
objective  and  quantitative  measurement  of  the  classification  process  and  are  used  to  supplement 
advanced  classification  objectives. 

The  seeds  for  this  demonstration  are  small  ISO80s  and  37mm  projectiles.  The  objective  for  the 
advanced  classification  process  for  this  demonstration  is  to  correctly  classify  100%  of  all  TOI. 

3.6.1  Metric 

The  metrics  for  this  objective  are  the  percentage  of  TOI  correctly  identified  on  the  TOI  lists. 

3.6.2  Data  Requirements 

Ranked  anomaly  lists,  separated  into  TOI  and  non-TOI  lists,  were  used  to  judge  the  success  of  this 
objective. 
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3.6.3  Success  Criteria 

The  objective  is  considered  to  be  met  if  100%  of  the  QC  seeds,  population  seeds,  and  native  TOI 
are  placed  on  the  TOI  list. 

3.7  OBJECTIVE:  CORRECTLY  IDENTIFY  GROUP 

The  objective  is  to  correctly  assign  each  TOI  and  non-TOI  to  either  the  small  group  (small  ISO80 
and  up  to  40mm  diameter),  medium  group  (medium  ISO  and  up  to  8  lnnn  diameter),  or  large  group 
(90mm  and  105mm  projectiles). 

3.7.1  Metric 

The  metrics  for  this  objective  are  the  percentage  of  TOI  and  non-TOI  correctly  grouped  in  either 
the  small,  medium,  or  large  groups. 

3.7.2  Data  Requirements 

Anomalies  grouped  as  small,  medium,  or  large  were  used  to  judge  the  success  of  this  objective. 
The  data  depended  on  the  usability  of  the  beta  (p2)  and  P3  polarizability  curves. 

3.7.3  Success  Criteria 

The  group  assignment  task  is  considered  successful  if  85%  or  more  of  the  group  designations  are 
correct. 

3.8  OBJECTIVE:  CORRECT  ESTIMATION  OF  EXTRINSIC  TARGET 
PARAMETERS 

This  objective  involves  the  accuracy  of  the  target  parameters  that  are  estimated  in  the  first  phase 
of  the  analysis  (data  inversion).  Successful  classification  is  possible  only  if  the  input  features  are 
internally  consistent.  The  obvious  way  to  satisfy  this  condition  is  to  estimate  the  various  target 
parameters  accurately. 

3.8.1  Metric 

Accuracy  of  estimation  of  extrinsic  target  parameters  is  the  metric  for  this  objective. 

3.8.2  Data  Requirements 

The  predicted  anomaly  locations  and  depths  were  compared  against  the  measured  locations  of 
items  recovered  during  intrusive  investigations. 

3.8.3  Success  Criteria 

The  objective  is  considered  a  success  if  the  estimated  X,  Y  locations  are  within  15  cm  (la)  and 
the  estimated  depths  (Z)  are  within  10  cm  (la). 
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3.9  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms,  was  able  to  classify  the  targets  with  high  accuracy.  This  objective 
concerns  the  component  of  the  classification  problem  that  involves  false  alann  reduction. 

Because  the  number  of  clutter  items  that  may  resemble  20mm  projectiles  is  unknown,  the  success 
metric  for  this  objective  (50%)  is  lower  than  that  of  most  previous  demonstrations,  which  typically 
use  a  metric  of  65%. 

3.9.1  Metric 

The  metric  for  this  objective  is  the  number  of  cued  anomalies  that  can  be  correctly  classified  as 
non-TOI. 

3.9.2  Data  Requirements 

WESTON  prepared  a  prioritized  non-TOI  list  from  the  cued  anomaly  list.  Institute  for  Defense 
Analysis  (IDA)  personnel  used  their  scoring  algorithms  to  assess  the  results. 

3.9.3  Success  Criteria 

The  objective  is  considered  a  success  if  more  than  50%  of  the  non-TOI  items  can  be  correctly 
labeled  as  non-TOI  while  meeting  the  objectives  or  success  criteria  for  TOI  stated  in  Table  3-2. 

3.10  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

Anomalies  for  which  reliable  parameters  cannot  be  estimated  cannot  be  classified  by  the  classifier. 
These  anomalies  must  be  placed  in  the  dig  category  and  will  consequently  reduce  the  effectiveness 
of  the  classification  process. 

3.10.1  Metric 

The  number  of  anomalies  for  which  reliable  parameters  cannot  be  estimated  is  the  metric  for  this 
objective. 

3.10.2  Data  Requirements 

Anomalies  for  which  parameters  cannot  be  reliably  estimated  are  assigned  a  category  0  on  the  final 
ranked  anomaly  list. 

3.10.3  Success  Criteria 

The  objective  is  considered  a  success  if  reliable  parameters  can  be  estimated  for  >  95%  of  the 
anomalies  on  each  sensor  anomaly  list. 
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4.  SITE  DESCRIPTION 


Andersen  AFB  is  a  United  States  Air  Force  base  spanning  20,000  acres  in  the  city  of  Yigo 
(pronounced  "Geego")  on  the  northern  end  of  the  island  of  Guam.  Along  with  Naval  Base  Guam, 
Andersen  AFB  was  placed  under  the  command  of  Joint  Region  Marianas  on  1  October  2009. 
Andersen  AFB,  36th  Wing,  Air  Mobility  Command,  opened  as  North  Field  in  1944,  and  was 
primarily  used  as  a  B-29  staging  base  in  the  Pacific  during  WWII.  Later  it  was  renamed  after 
Brigadier  General  James  R.  Andersen,  fonner  Chief  of  Staff  for  the  Army  Air  Force,  Pacific.  The 
base  continues  to  support  strategic  operations  in  the  region,  and  serves  as  a  staging  base  for 
activities  in  Asia  and  the  South  Pacific.  The  bulk  of  Andersen’s  duties  since  WWII  have  been  as  a 
Strategic  Air  Command  base,  supporting  activities  in  Korea  and  Vietnam. 

4.1  SITE  SELECTION 

The  Anderson  AFB  NRP  area  was  chosen  as  the  first  geophysical  classification  demonstration  site 
located  in  Guam.  The  NRP  area  will  be  undergoing  construction  activities  under  the  MILCON 
program.  MEC  are  known  or  suspected  to  be  present  at  various  sites  on  Guam  as  a  result  of  WWII 
battles  and  subsequent  military  activities.  MEC  are  a  safety  hazard  and  may  constitute  an  imminent 
and  substantial  endangerment  to  construction  personnel  and  the  local  population.  As  a  requirement 
of  the  MILCON  program,  sites  such  as  the  NRP  area  that  have  a  moderate  to  high  probability  of 
encountering  MEC  require  a  removal  action  in  advance  of  construction.  This  demonstration  was 
integrated  with  a  previously  scheduled  MEC  removal  action. 

The  NRP  area  site  provides  opportunities  to  demonstrate  the  capabilities  and  limitations  of  the 
geophysical  classification  process  on  a  MILCON  project  footprint  crowed  with  utilities,  previous 
infrastructure,  and  potential  MEC.  Results  of  the  demonstration  provide  insight  and  consideration 
for  future  use  of  geophysical  classification  ahead  of  MILCON  programs  and  projects.  Also, 
besides  the  MILCON  efforts,  there  are  a  number  of  MMRP  sites  on  Guam  that  will  benefit  from 
the  results  of  this  demonstration. 

4.2  BRIEF  SITE  HISTORY 

Guam  was  an  American  territory  in  1941  when  the  island  was  invaded  by  the  Japanese  military  on 
December  8,  1941.  The  Japanese  military  occupation  of  Guam  lasted  from  1941  until  1944  when 
the  United  States  military  liberated  Guam.  The  Battle  of  Guam  began  on  July  21,  1944  with 
American  troops  landing  on  the  western  side  of  the  island.  After  several  weeks  of  heavy  fighting, 
Japanese  forces  officially  surrendered  on  August  10,  1944.  The  heavy  military  activity  on  Guam 
caused  a  variety  of  American  and  Japanese  war  time  remnants,  including  MEC,  to  be  distributed 
throughout  the  island.  The  results  of  the  MEC  distribution  has  resulted  in  the  investigation  and 
removal  of  MEC  in  a  systematic  process  under  the  MILCON  program. 
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4.3  MUNITIONS  CONTAMINATION 


The  expected  munitions  at  Andersen  AFB  NRP  area  are  listed  in  Table  4-1,  which  are  based  on 
the  infonnation  in  the  Explosives  Safety  Submission  (ESS)  and  the  current  conceptual  site  model. 
The  known  items  in  the  table  identify  munitions  types  recovered  by  Explosive  Ordnance  Disposal 
(EOD)  Units  between  1991  and  2011  and  other  suspected  items. 

Table  4-1.  Known  and  Suspected  Munitions  Types 

Anderson  AFB,  NRP  Area 

MK II  Hand  Grenades* 

20mm  mortar 
60mm  mortar* 

8 1  mm  mortar* 

105mm  projectiles* 

155mm  projectile* 

5- inch  projectile* 

6- inch  projectile* 

100  lb  bomb* 

*  Items  were  recovered  by 
EOD  Units  between  1991  and  2011. 


4.4  SITE  CONFIGURATION 

The  demonstration  area  covers  3  acres  within  the  NRP  area.  The  TEMTADS  2x2  was  used  in 
dynamic  mode  to  survey  the  3  acre  site  with  100%  coverage.  The  demonstration  site  boundaries 
and  surface  types  are  shown  in  Figure  4-1. 
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Figure  4-1.  NRP  TEMTADS  2x2  Demonstration  Area 
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5.  TEST  DESIGN 

5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  objective  of  this  program  was  to  demonstrate  a  methodology  for  the  use  of  classification  in 
the  munitions  response  process.  The  three  key  components  of  this  methodology  were  collection  of 
high-quality  advanced  sensor  dynamic  detection  mapping  data;  selection  of  anomalous  regions  in 
those  data;  and  subsequent  cued  interrogation  and  analysis  of  the  selected  anomalies  using  physics- 
based  models  to  extract  target  parameters  such  as  size,  shape,  and  materials  properties;  and  the  use 
of  those  parameters  to  construct  a  ranked  anomaly  list.  Each  of  these  components  was  handled 
separately  in  this  program. 

Dynamic  data  were  processed,  and  anomalies  were  selected  and  combined  with  existing 
EM61-MK2  datasets.  Each  of  the  selected  anomalies  were  subsequently  cued  using  the 
TEMTADS  2x2.  Individual  cued  data  sets  were  processed  using  existing  routines  in  UX-Analyze 
Advanced  to  extract  target  parameters.  These  parameters  were  passed  to  the  classification  routines 
that  were  used  to  produce  ranked  anomaly  lists. 

A  total  of  1,195  anomalies  in  the  NRP  area  were  selected  for  cued  interrogation.  Intrusive 
investigation  of  these  anomalies  was  handled  by  the  contractor  perfonning  the  removal  action. 

5.2  SITE  PREPARATION 

5.2.1  Survey  of  Historical  Records 

Much  of  the  historical  infonnation  on  this  site  has  been  collected  in  the  ESS.  This  report  is  posted 
on  the  ESTCP  ftp  server  and  can  be  used  for  reference. 

5.2.2  First-Order  Navigation  Points 

Two  first-order  survey  monuments  were  installed  at  the  site.  Their  labels  and  coordinates  are 
provided  in  Table  5-1. 


Table  5-1.  Geodetic  Control  Locations 


ID 

Latitude 

Longitude 

Elevation 

NAVD88 

(m) 

Northing  (m) 

Easting  (m) 

ESTCP 1 

N13°35’43. 667395 

E144°55’49. 582789 

174.59 

1503938.212 

276077.554 

ESTCP2 

N13°35’25. 008458 

E144°55 ’25. 235422 

160.518 

1503370.904 

275340.62 

5.2.3  Initial  EMI  Survey 

Anomaly  density  information  derived  from  previously  collected  full  coverage  EM61-MK2 
dynamic  survey  data  was  used  to  define  the  demonstration  site  boundaries.  The  TEMTADS  2x2 
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demonstration  area  included  a  portion  in  an  open  grassy  area,  as  well  as  a  portion  on  the  adjacent 
runway  tannac. 

5.2.4  Surface  Sweep 

The  contractor  perfonning  the  removal  action  conducted  a  surface  sweep  across  the  demonstration 
area  prior  to  collecting  the  dynamic  survey  data.  UXO  technicians  performed  the  surface  sweep  to 
remove  surface  metal  and  any  explosive  hazards  associated  with  potential  MEC. 

5.2.5  Seed  the  Site 

WESTON  emplaced  36  seed  items  consisting  of  inert  37mm  projectiles  and  Small  Schedule  80 
ISOs  within  the  demonstration  area  in  accordance  with  the  parameters  laid  out  in  the  ESTCP  MR 
Live  Site  Demonstration  Seeding  Plan.  However,  due  to  time  constraints  encountered  during  the 
dynamic  detection  survey,  only  19  seed  items  fell  within  the  final  3 -acre  coverage  area  surveyed 
during  with  the  TEMTADS  2x2.  During  the  seeding  process,  each  flagged  location  was  swept  with 
a  Schonstedt  to  ensure  a  clean  area  for  emplacement.  A  hole  was  dug  and  seeds  were  placed  at  the 
appropriate  depth  based  on  seed  type,  with  larger  items  placed  at  greater  depth.  Physical 
characteristics  of  the  seed  were  recorded  on  a  whiteboard  and  placed  alongside  the  excavated  hole 
for  a  photograph.  A  Trimble  R8  Real  Time  Kinematic  (RTK)  GPS  was  used  to  measure  and  record 
the  location  of  each  anomaly  at  depth. 

5.2.6  Establish  an  IVS  and  Test  Stand 

An  IVS  and  test  stand  area  were  established  within  a  quiet  area  free  of  subsurface  metal  adjacent 
to  the  demonstration  site.  The  IVS  was  visited  twice  daily  to  verify  proper  sensor  operation  and 
functionality.  An  as-built  schematic  of  the  IVS  is  detailed  in  Figure  5-2.  Details  of  seed  items 
placed  in  the  IVS  are  listed  in  Table  5-2. 
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Figure  5-1.  Layout  of  the  IVS  Established  at  the  NRP  Area 
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Table  5-2.  Details  of  the  Instrument  Verification  Strip 


Item  ID 

Description 

Design 
Easting  (m) 

Design 

Northing 

(m) 

Depth  (m) 

Inclination 

Azimuth 

IVS01 

Small  ISO40 

276009.309 

1503311.042 

0.10 

Horizontal 

Along-Track 

IVS02 

Small  ISO40 

276006.455 

1503309.922 

0.10 

Vertical 

N/A 

IVS03 

Small  ISO40 

276003.791 

1503308.678 

0.23 

Horizontal 

Along-Track 

IVS04 

Small  ISO40 

276001.055 

1503307.465 

0.23 

Vertical 

N/A 

IVS05 

Medium  ISO 

275998.11 

1503306.411 

0.43 

Horizontal 

Along-Track 

IVS06 

Medium  ISO 

275995.119 

1503305.304 

0.43 

Vertical 

N/A 

IVS07 

Large  ISO 

275992.423 

1503304.063 

0.44 

Vertical 

N/A 

IVS08 

Shotput 

275989.610 

1503302.982 

0.30 

N/A 

N/A 

Due  to  limited  suitable  space  available  to  install  a  separate  IVS  within  the  NRP  area,  the  existing 
IVS  established  for  the  EM61-MK2  survey  being  perfonned  in  support  of  the  MEC  removal 
activities  was  used  for  the  demonstration.  The  IVS  was  modified  by  adding  a  shotput.  However, 
the  existing  seeds,  which  included  Small  Schedule  40  ISOs,  were  left  unchanged. 
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The  IVS  was  used  for  daily  function  checks  of  the  survey  equipment.  WESTON  surveyed  the  strip 
twice  daily,  once  each  morning  and  evening  of  survey  work,  in  order  to  monitor  the  responses  and 
detected  positions  of  the  IVS  seed  items  throughout  the  duration  of  the  project. 

A  test  stand  location  was  established  near  the  IVS  at  a  quiet  location  free  of  subsurface  metal  and 
was  used  to  measure  the  signatures  of  TOI  expected  to  be  present  within  the  demonstration  area. 
The  only  native  TOI  available  during  the  demonstration  were  20mm  and  37mm  projectiles,  so  test 
stand  measurements  were  limited  to  those  items.  Measurements  were  performed  for  each  test  item 
at  multiple  depths  and  orientations.  The  test  stand  data  collected  are  listed  in  Table  5-3. 


Table  5-3.  Test  Stand  Items  and  Orientations 


Item  ID 

Depth  (cm) 

Orientation 

Small  ISO80 

10,  20 

Horizontal,  Vertical 

20mm 

5,  10 

Horizontal,  Vertical 

37mm  projectile 

10,  15,20 

Horizontal,  Vertical 

5.3  DATA  COLLECTION 
5.3.1  Dynamic  Data  Collection 

WESTON  performed  dynamic  detection  surveys  with  the  TEMTADS  2x2  using  EM3DAcquire  to 
control  data  acquisition  parameters,  storage  of  data,  and  real-time  monitoring  of  TEMTADS  2x2 
sensor  and  peripheral  IMU  and  RTK  data  streams.  A  Trimble  R8  RTK  GPS  was  used  for 
navigation.  The  rover  head  was  mounted  directly  over  the  center  of  the  TEMTADS  2x2  array.  An 
IMU  was  installed  directly  below  the  rover  head  to  capture  pitch,  roll,  and  yaw  of  the  sensor. 
Dynamic  TEMTADS  2x2  survey  data  were  acquired  with  a  design  line  spacing  of  0.50  m. 

Dynamic  detection  surveys  were  conducted  over  the  course  of  5  days  from  January  7  to  17,  2014. 
There  were  a  number  of  no-collection  days  during  this  timeframe  due  to  inclement  weather  as  well 
as  TEMTADS  2x2  sensor  and  computer  malfunctions.  Dynamic  detection  data  were  collected  over 
a  total  of  2.97  acres  using  the  TEMTADS  2x2,  equating  to  an  average  of  0.6  acres  per  day  of  data 
collection.  Data  gaps  were  typically  identified  in  the  field  and  re-collected  the  same  day  to  ensure 
full  coverage. 

Dynamic  detection  coverage  for  both  the  TEMTADS  2x2  and  the  existing  EM61-MK2  dataset  is 
shown  in  Figure  5-2.  The  locations  of  the  final  targets  derived  from  the  TEMTADS  2x2  and 
EM61-MK2  datasets,  as  well  as  the  subsequent  cued  measurement  locations  for  each  reacquired 
target  location,  are  also  shown. 
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Figure  5-2.  NRP  Area  TEMTADS  2x2  and  EM61-MK2  Geophysical  Survey  Coverage 
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5.3.2  Dynamic  Data  Processing 


The  raw  binary  TEMTADS  2x2  *.TEM  files  were  converted  to  ASCII  *.CSV  files  using  NRL 
TEMT2CSV  data  conversion  software.  Converted  data  were  then  imported  into  Geosoft  Oasis 
montaj  for  processing  and  analysis  using  scripted  import  routines.  Upon  import,  raw  data  were 
inspected  to  ensure  that  sensor  data  were  valid  and  that  peripheral  input  data  streams  (GPS,  IMU) 
were  present  and  valid.  Dynamic  detection  data  were  imported,  processed,  and  validated  on  a  daily 
basis. 

Data  from  each  TEMTADS  2x2  sensor  were  then  located  using  the  UX- Analyze  Advance  “Create 
Located  Database”  GX  and  exported  to  a  separately  located  database,  with  each  sensor  assigned  a 
unique  version  number  per  line  (e.g.  Line  1,  sensor  1  equates  to  Line  1.1,  Line  1,  sensor  2  equates 
to  Line  1 .2).  Sensor  offsets  were  calculated  in  reference  to  the  RTK  GPS  position  at  the  center  of 
the  array,  with  IMU  data  used  to  adjust  for  pitch,  roll,  and  yaw  in  the  sensor  array.  Data  analysis 
and  anomaly  selection  were  performed  on  the  z-axis  component  of  the  four  receiver  cubes. 

The  dynamic  detection  data  were  then  levelled  using  a  de-median  background  removal  filter.  Once 
the  daily  data  had  been  imported,  validated,  and  levelled,  the  data  were  then  merged  into  a  master 
site  database  containing  all  dynamic  data  collected  to  date.  Once  data  collection  was  complete,  the 
master  database  was  used  for  gridding,  anomaly  selection,  and  analysis. 

5.3.3  Anomaly  Selection 

Anomalies  were  selected  from  processed  TEMTADS  2x2  dynamic  detection  data  using  the 
Geosoft  Blakely  grid  peak  detection  algorithm.  To  determine  a  suitable  anomaly  selection 
threshold,  dynamic  test  data  were  acquired  over  a  20mm  projectile  placed  adjacent  to  the  NRP 
demonstration  area  in  a  horizontal  orientation  at  four  depths  -  5cm,  10cm,  15cm,  and  20cm.  At 
each  depth,  dynamic  TEMTADS  measurements  were  collected  along  individual  passes  with  the 
test  item  oriented  both  along-track  and  cross-track.  Minimum  responses  for  each  20mm  depth  are 
listed  in  Table  5-4. 


Table  5-4.  Results  of  20mm  Dynamic  TEMTADS  2x2  Measurements 


Item 

Depth  (cm) 

Orientation 

Alignment 

Peak  Response  (mV/A) 
Sum  of  Gates  5-12 

20mm 

5 

Horizontal 

Along-Track 

17.18 

20mm 

10 

Horizontal 

Along-Track 

9.98 

20mm 

15 

Horizontal 

Along-Track 

5.17 

20mm 

20 

Horizontal 

Along-Track 

4.60 

Sub-window  statistic  calculations  were  then  perfonned  on  data  from  the  IVS  background  strip,  as 
well  as  at  four  anomaly  free  locations  within  the  dynamic  TEMTADS  2x2  dataset  to  assess 
background  noise.  RMS  noise  varied  from  0.77mV/A  RMS  to  1.97mV/A  RMS  based  on  location 
within  the  NRP  area.  Data  from  each  area  was  combined  and  analyzed,  and  the  results  are  listed 
in  Table  5-5. 
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Table  5-5.  NRP  Area  Background  Statistics 


Minimum 
Response  (mV/A) 

Maximum 
Response  (mV/A) 

Mean  Response 
(mV/A) 

RMS  Noise 
(mV/A) 

7xRMS  Noise 
(mV/A) 

-6.69 

4.34 

-0.19 

0.99 

6.94 

RMS  noise  in  anomaly  free  areas  within  NRP  area  were  approximately  0.99mV/A  for  the 
monoZ_5_12_lev  data  channel  (sum  of  gates  5  -  12).  A  threshold  of  6.94mV/A  (7  times  the  RMS 
noise  level)  was  used  to  allow  the  selection  of  a  20mm  at  a  depth  of  up  to  10  cm  below  ground 
surface,  and  potentially  deeper,  depending  on  item  orientation. 

A  preliminary  target  list  was  generated  from  the  TEMTADS  2x2  dataset  based  on  the  6.94  mV/A 
threshold.  The  TEMTADS  2x2  anomaly  locations  were  then  compared  to  the  target  locations  in 
the  existing  EM61-MK2  target  list  to  generate  a  final  target  list  of  coincident  TEMTADS  2x2  and 
EM61-MK2  anomalies.  The  locations  for  cued  reacquisition  used  the  TEMTADS  2x2  target 
locations  in  areas  with  dynamic  TEMTADS  2x2  data  coverage,  and  the  EM61-MK2  anomaly 
locations  in  areas  with  only  EM61-MK2  data  coverage.  A  total  of  970  anomalies  were  selected 
from  the  TEMTADS  2x2  dataset.  An  additional  225  anomalies  were  selected  from  the  existing 
EM61-MK2  dataset  collected  for  the  MEC  removal  action. 

5.3.4  Cued  Data  Collection 

WESTON  performed  cued  data  collection  over  the  course  of  10  days  between  January  18  and 
February  05,  2014  at  1,195  anomalies  that  were  based  on  the  approved  target  lists.  Cued  data 
collection  averaged  120  cued  locations  each  day  during  the  10  field  days.  Cued  target  locations 
were  reacquired  with  the  RTK  GPS  and  flagged  each  day  prior  to  data  collection.  The  operator 
then  positioned  the  TEMTADS  2x2  within  40  cm  of  the  center  of  flagged  location  and  collected  a 
cued  measurement  over  the  anomaly.  To  account  for  changing  background  conditions,  background 
measurements  were  collected  once  per  hour  in  a  quiet  area  identified  in  the  dynamic  data  set.  The 
cued  data  were  reviewed  each  evening,  and  cued  locations  that  fell  outside  the  40-cm  offset  metric 
were  re-collected  as  necessary. 

5.3.5  Cued  Data  Processing 

Cued  data  processing  was  perfonned  using  the  UX-Analyze  Advanced  extension  in  Geosoft  Oasis 
montaj.  Cued  background  data  were  imported  and  qualitatively  verified,  with  any  outliers  removed 
from  the  background  dataset.  After  background  data  had  been  verified,  cued  anomaly  data  were 
imported,  verified  for  completeness,  and  background  corrected  using  the  cued  background  data 
spatially  and  temporally  closest  to  the  cued  anomaly  location. 

Inversions  were  performed  on  each  cued  anomaly  using  both  single-source  and  multi-source 
models  to  extract  target  parameters,  fit  coherence,  and  predicted  locations  and  depths  for  each 
model.  The  primary  parameters  used  for  classification  were  the  three  polarizabilities  (Pi,  P2,  and 
P3)  calculated  for  each  single-source  and  multi-source  modeled  result. 


MR-201231 


20 


4/27/2016 


Daily  quality  control  was  performed  on  the  cued  anomaly  data  in  which  the  cued  location  (Real 
Time  Kinematic  [RTK]  GPS  location),  modeled  locations,  and  flagged  locations  were  compared 
to  verify  that  the  center  of  the  TEMTADS  2x2  array  was  within  the  40-cm  radius  of  the  anomaly 
source.  Targets  outside  the  40-cm  metric  were  identified  and  re-collected  as  time  allowed. 

After  the  individual,  inverted  locations  from  the  cued  sensors  were  complete,  the  data  were 
combined  into  a  master  dig  list  for  each  anomaly  group.  These  Master  Dig  Lists  contained  one 
entry  for  each  predicted  anomaly  from  the  inversions  of  the  cued  data. 

5.3.6  Data  Handling 

WESTON  provided  dynamic  detection  data  to  the  ESTCP  Program  Office  for  archiving  in  raw 
instrument  *.TEM  and  converted  ASCII  *.CSV  formats,  as  well  as  located,  processed  data  in 
Geosoft  database  (*.gdb)  fonnat.  Cued  data  were  provided  in  raw  instrument  files  *.TEM  format, 
uncorrected  ASCII  *.CSV  files,  and  background  corrected  data  in  Geosoft  *.GDB  fonnat. 

5.4  INTRUSIVE  ACTIVITY  AND  PROCEDURES 

Intrusive  investigation  was  perfonned  by  the  3rd-party  contractor  performing  the  MEC  removal 
action  and  construction  support  work  at  the  NRP  area  MILCON  project.  Intrusive  procedures  were 
perfonned  to  the  specifications  of  the  3rd-party  contractor’s  work  plan,  and  did  not  follow  the 
standard  ESTCP  intrusive  investigation  requirements,  so  dig  data  obtained  did  not  include  precise 
locations,  depths,  nor  descriptions. 
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6.  CLASSIFICATION 

Classification  for  cued  anomaly  data  collected  at  the  NRP  Area  was  based  primarily  on  the  library 
match  statistic  generated  in  UX-Analyze  Advanced  during  library  matching  of  modeled  results 
against  the  demonstration  site  TOI  library  of  expected  munitions  and  TOI.  The  multi-criteria 
method  (introduced  in  UX-Analyze  v8.2)  was  used,  in  which  the  library  match  statistic  of  four 
combinations  of  beta  ((3)  criteria  weights  were  calculated  and  then  averaged  to  create  a  decision 
statistic.  Library  match  statistic  combinations  used  to  calculate  the  decision  statistic  are  listed  in 
Table  6-1. 

Table  6-1.  Multi-Criteria  Library  Match  Statistic  Combinations 


Library  Match 

Criteria  Weighting 

Statistic 

Combination 

PI 

P2 

P3 

LmStat  111 

1 

1 

1 

LmStat  110 

1 

1 

0 

LmStat  Oil 

0 

1 

1 

LmStat  100 

1 

0 

0 

In  addition  to  the  matching  of  [3s  to  a  library  of  known  TOI,  self-matching  was  also  performed  in 
which  Ps  for  each  modeled  result  were  compared  to  the  Ps  of  all  other  modeled  results  to  identify 
clusters  of  similar  items  that  may  not  be  present  in  the  TOI  library.  Items  identified  in  self-match 
clusters  or  from  clusters  identified  in  feature  space  (size  vs  decay  plot)  were  then  evaluated  to  be 
included  in  a  training  data  request.  Cluster  analysis  resulted  in  a  training  data  request  for  13  digs. 
The  data  received  from  the  training  digs  were  used  to  further  refine  the  decision  metric  thresholds 
in  the  ranked  anomaly  list.  Clutter  items  from  the  training  dig  results  were  also  added  to  the  clutter 
library  to  be  used  in  the  final  classification  process. 

The  following  parameters  were  used  in  the  ordering  of  the  ranked  anomaly  list: 

•  Decision  statistic 

•  Signal  amplitude 

•  Fit  depth 

•  Size  and  decay 

•  Array  to  fit  location  offset 

•  Fit  coherence 

•  Library  match  statistic  to  clutter  library 

From  these  parameters,  each  single  and  multi-source  modeled  result  was  placed  into  one  of  four 
categories  0  to  3,  with  the  best  ranked  modeled  result  from  each  cued  anomaly  being  passed  onto 
to  a  final  ranked  list  based  on  the  parameters  outlined  in  Table  6-2. 
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Table  6-2.  Ranked  List  of  Parameters 


Ranked  List  Category 

Category  Description 

Criteria 

Category  -1 

Training  Digs 

Training  digs  requested  by  analyst 

Category  0 

Cannot  Analyze 

Fit  coherence  <  0.8 

Category  1 

Likely  TOI 

Decision  statistic  >0.90 

Category  2 

Cannot  Decide 

Decision  statistic  >0.85  but  <0.90 

Category  3 

Clutter 

Decision  statistic  <0.85 

The  ranked  anomaly  list  derived  from  this  classification  scheme  resulted  in  the  classification 
distribution  displayed  in  Figure  6-1. 

Figure  6-1.  NRP  Area  Classification  Distribution 
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WESTON  submitted  the  ranked  anomaly  list  to  the  ESTCP  Program  Office  for  scoring  by  the 
IDA.  The  intrusive  results,  including  anomaly  descriptions,  and  a  receiver  operating  characteristic 
(ROC)  curve  indicating  the  percentage  of  TOI  identified  for  the  243  anomalies  placed  on  the  dig 
list  (Categories  0,  1,  and  2),  were  supplied  to  WESTON  for  review. 
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7.  PERFORMANCE  ASSESSMENT 


The  performance  objectives  for  this  demonstration  and  the  corresponding  results  are  summarized 
in  Table  7-1  and  Table  7-2. 

Table  7-1  lists  the  performance  objectives  for  the  field  activities.  These  apply  to  the  detection  and 
classification  work  perfonned  at  NRP  Area.  Table  7-2  lists  the  perfonnance  objectives  for  the 
advanced  classification  activities.  These  apply  to  the  similar  work  perfonned  using  NRP 
demonstration  area  advanced  classification  data.  Results  evaluations  are  provided  in  the  following 
sections. 


Table  7-1.  Performance  Objectives  and  Results  for  Field  Activities 


Performance 

Objective 

Metric 

Success  Criteria 

Results 

Repeatability  ofIVS 
measurements 

Amplitude  of  EM 
anomaly 

Measured  target 
locations 

Adv.  Sensors  Survey: 

Down-track  location  ±25  cm 

Adv.  Sensors  Cued: 

Library  match  >90%  using  3- 
criterion  metric  with  equal 
weighting  to  the  three  criteria  using 
first  day’s  IVS  inversion  as  the 
library  item. 

Fail  -  Two  (2)  IVS  events 
exceeded  the  detection 
offset  of  <25  cm 

Fail  -  Only  87%  of  cued 

IVS  events  achieved  a 
>90%  library  match  using 
an  equally  weighted  3- 
criterion  match 

Explanation  provided  in 
subsequent  sections. 

Complete  coverage 
of  the  demonstration 
site 

Footprint  coverage 
calculated  using 
UX-Process 
Footprint 

Coverage  QC  tool; 
excludes 

inaccessible  areas. 

>=  85%  coverage  at  0.50-m  line 
spacing;  and 

>=  98%  coverage  at  0.60-m  line 
spacing 

Pass  -  93.8%  coverage  was 
achieved  at  a  0.50-m  line 
spacing,  99.1%  at  0.60  m 
line  spacing 

Along-line 

measurement 

spacing 

Point-to-point 
spacing  from  data 
set 

98%  <  25-cm  along-line  spacing 

Pass  -  100%  of  the  along- 
line  spacing  was  <25  cm 

Detection  of  all  TOI 

Percent  detected  of 
TOI 

100%  of  TOI  detected  within  40-cm 
halo  of  the  surveyed  location 

Pass  -  100%  of  TOI  was 
detected  within  a  40-cm 
halo 

Cued  interrogation 
of  anomalies 

Instrument 

position 

100%  of  anomalies  where  the  center 
of  the  instrument  is  positioned 
within  40  cm  of  actual  target 
location 

Fail  -  Only  94%  of  the  cued 
measurements  were  within 
the  40-cm  metric 

Explanation  provided  in 
subsequent  sections. 
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Table  7-2.  Performance  Objectives  and  Results  for  Advanced  Classification  Activities 


Performance 

Objective 

Items 

Metric 

Success  Criteria 

Results 

Correctly  classify 

QC  seeds  and 
correctly  classify 
native  and 
population  seed 
items 

All  seeds  and  all 
native  TOI 

Percent  classified 
as  TOI 

100%  classified  as 

TOI 

Pass  -  all  TOI  were 
properly  classified 

Correctly  identify 
group 

All  TOI  and  all 
excavated  Non 

TOI 

Percent  of  TOI  and 
excavated  Non 

TOI  grouped 
correctly 

85%  correctly 
grouped  in  the 

Small,  Medium  and 
Large  groups 

Pass  -  98%  were 
assigned  to  the  correct 
group 

Correct  estimation 
of  extrinsic  target 
parameters 

All  excavated 
anomalies 

Measured  location 
and  depth  to  center 
of  mass  of 
recovered  items 

X,  Y<  15  cm  (la) 

Z  <  10  cm  (la) 

Pass-  a  ofX,Y 
offsets  <15  cm 

Pass-  a  of  Z  <10  cm 
of  the  actual  depth 

Explanation  provided 
in  subsequent 
sections. 

Maximize  correct 
classification  of 
non-TOI 

All  non-TOI 

Number  of  false 
alarms  eliminated 

Reduction  of  clutter 
digs  by  >50%  while 
meeting  all  other 
demonstration 
objectives 

Pass  -  81%  of  non- 
TOI  were  correctly 
classified 

Minimize  number  of 
anomalies  that 
cannot  be  analyzed 

All  cued 
anomalies 

Number  of 
anomalies  that 
must  be  classified 
as  “Unable  to 
Analyze” 

Reliable  target 
parameters  can  be 
estimated  for  >  95% 
of  anomalies  on  each 
sensor’s  anomaly 
list. 

Pass  -  only  2% 
classified  as  “Cannot 
Analyze” 

7.1  OBJECTIVE:  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  STRIP 
MEASUREMENTS 

7.1.1  Dynamic  IVS 

This  objective  involved  the  repeatability  of  the  detection  location  of  seed  items  in  dynamic  IVS 
data  collection.  Seed  item  offsets  for  each  dynamic  IVS  data  collection  event  were  tracked 
throughout  the  life  of  the  dynamic  portion  of  the  project.  This  objective  was  considered  to  be  met 
if  all  locations  of  seed  items  as  detected  in  the  dynamic  IVS  data  were  offset  <25  cm  from  the 
actual  surveyed  location.  Results  for  dynamic  detection  IVS  surveys  are  detailed  in  Table  7-3. 
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Table  7-3.  IVS  Seed  Item  Detection  Results 


Seed  Item 

Seed  Type 

Minimum  Offset 
(cm) 

Maximum  Offset 
(cm) 

Average  Offset 
(cm) 

IVS01 

Small  ISO40 

0.06 

0.20 

0.14 

IVS02 

Small  ISO40 

0.02 

0.18 

0.10 

IVS03 

Small  ISO40 

0.16 

0.27 

0.23 

IVS04 

Medium  ISO 

0.07 

0.20 

0.12 

IVS05 

Medium  ISO 

0.16 

0.26 

0.20 

IVS06 

Medium  ISO 

0.02 

0.16 

0.08 

IVS07 

Large  ISO 

0.03 

0.19 

0.11 

IVS08 

Shotput 

0.05 

0.18 

0.11 

The  25cm  metric  was  exceeded  on  2  occasions;  once  at  seed  IVS03  and  once  at  seed  IVS05.  The 
remainder  of  the  detection  offsets  were  all  within  25  cm  of  the  actual  surveyed  locations,  however, 
as  a  result  of  the  two  exceedances,  this  objective  was  not  met. 

7.1.2  Cued  IVS 

This  objective  involved  the  repeatability  of  classification  of  IVS  seed  items  during  cued  data 
collection.  Seed  item  library  match  statistics  for  each  cued  IVS  data  collection  event  were  tracked 
throughout  the  life  of  the  cued  portion  of  the  project.  This  objective  was  considered  to  be  met  if 
the  library  match  statistic  for  all  seed  items  cued  in  the  IVS  was  >90%  when  using  a  three-criterion 
metric  with  equal  weighting  to  the  three  criteria  when  measured  against  the  first  day’s  cued  IVS. 
Results  for  cued  IVS  surveys  are  detailed  in  Table  7-4. 

This  performance  objective  was  not  met,  as  not  all  of  the  library  match  statistics  of  the  inverted 
data  to  the  IVS  seed  item  library  were  >90%  during  the  duration  of  the  cued  survey. 

The  IVS  results  were  monitored  throughout  the  duration  of  the  cued  data  collection,  and  data  not 
meeting  the  perfonnance  objective  were  reviewed  real-time  in  an  attempt  to  determine  the  source 
of  any  noted  failures.  The  exact  source  of  the  failures  was  not  detennined,  however,  a  contributing 
factor  is  most  likely  the  IVS  construction  or  the  seed  items  installed  in  the  IVS.  Due  to  limited 
suitable  space  available  to  install  an  IVS  within  the  NRP  area,  the  existing  IVS  established  for  the 
EM61-MK2  survey  being  perfonned  in  support  of  the  clearance  activities  was  used  for  the 
demonstration.  Schedule  40  small  ISOs  were  used  in  the  IVS,  which  may  have  been  a  contributing 
factor  to  the  inconsistent  results  of  the  deeper  small  ISO  seed  item  IVS03.  It  is  unclear  what  caused 
the  match  statistic  deviations  over  the  medium  ISO  seed  items. 
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Table  7-4.  IVS  Library  Match  Results 


Seed  Item 

Seed  Type 

Seed  Depth  (m) 

Minimum  Library 
Match  Statistic 

Average  Library 
Match  Statistic 

IVS01 

Small  ISO40 

0.10 

0.97167 

0.991439 

IVS02 

Small  ISO40 

0.10 

0.9738 

0.988419 

IVS03 

Small  ISO40 

0.23 

0.861 

0.94605 

IVS04 

Medium  ISO 

0.23 

0.74866 

0.886833 

IVS05 

Medium  ISO 

0.43 

0.89001 

0.924231 

IVS06 

Medium  ISO 

0.43 

0.79654 

0.875971 

IVS07 

Large  ISO 

0.44 

0.95062 

0.978406 

IVS08 

Shotput 

0.30 

0.97599 

0.98819 

To  assess  data  usability,  production  data  were  evaluated  from  any  day  that  IVS  deviations  were 
observed,  and  no  indications  of  a  system  failure  with  respect  to  data  usability  were  observed.  Seed 
items  cued  during  the  days  in  question  were  properly  classified  with  high  confidence  statistical 
matches,  and  no  deficiencies  were  noted  in  sensor  performance. 

7.2  OBJECTIVE:  COMPLETE  COVERAGE  OF  THE  DEMONSTRATION  SITE 

This  objective  measured  the  effectiveness  of  the  dynamic  detection  survey  as  a  function  of  the 
amount  of  coverage  of  the  demonstration  area  by  the  TEMTADS  2x2  sensor.  This  objective  was 
considered  to  be  met  if  the  dynamic  detection  survey  achieved  85%  coverage  of  the  site  at  a  0.50- 
m  lane  spacing,  and  98%  of  the  site  at  0.60-m  lane  spacing.  The  UX-Process  Footprint  Coverage 
QC  tool  was  used  to  analyze  the  georeferenced  positions  of  the  center  of  the  TEMTADS  2x2 
sensor  array.  Data  were  collected  at  a  0.50m  lane  spacing  to  eliminate  gaps  caused  by  ruts  and 
rough  terrain.  This  objective  was  met,  as  93.8%  of  the  site  was  covered  at  a  0.50m  lane  spacing, 
and  99.1%  at  0.60m  lane  spacing. 

7.3  OBJECTIVE:  ALONG-LINE  MEASUREMENT  SPACING 

This  objective  evaluated  the  along-line  data  density,  or  sample  separation,  of  the  TEMTADS  2x2 
dynamic  detection  dataset  acquired  within  the  NRP  area.  The  metric  for  this  objective  was  the 
point-to-point  distance  as  measured  using  UX-Process  Sample  Separation  utility.  This  objective 
was  considered  to  be  met  if  98%  of  the  data  had  an  along-line  spacing  of  25  cm  or  less. 

The  UX-Process  Sample  Separation  tool  was  used  to  analyze  the  along-line  spacing  of  the 
georeferenced  data  positions  of  the  TEMTADS  2x2  sensor  array.  This  objective  was  met,  because 
100%  of  the  data  had  a  sample  separation  of  25  cm  or  less. 
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7.4  OBJECTIVE:  DETECTION  OF  ALL  TOI 


This  objective  evaluated  the  dynamic  detection  capabilities  of  the  TEMTADS  2x2  array.  The 
metric  for  this  objective  was  considered  to  be  met  if  100%  of  native  and  non-native  TOI  were 
detected  within  a  40-cm  halo  of  their  recorded  locations.  Non-native  TOI  within  the  TEMTADS 
2x2  demonstration  area  included  13  blind  seed  items  which  were  completely  blind  to  the  data 
collection  and  processing  teams,  as  well  as  6  QC  seed  items  in  which  the  locations  were  known. 

This  objective  was  met,  because  all  TOI  were  successfully  detected  within  the  TEMTADS  2x2 
dataset  within  40  cm  of  the  recorded  locations.  TOI  included  19  seed  items  (non-native  TOI) 
installed  by  WESTON  prior  to  the  dynamic  detection  survey.  TOI  detection  results  are  detailed  in 
Table  7-5.  For  comparison,  the  offset  from  the  existing  EM61-MK2  datasets  are  also  included  in 
the  table. 


Table  7-5.  TOI  Detection  Results 


Seed  ID 

Seed  Type 

Seed  Offset  (cm) 

TEMTADS  2x2 

EM61-MK2 

B  SI-01 

Small  ISO80 

5 

47 

B  SI-02 

Small  ISO80 

11 

23 

B  SI-03 

Small  ISO80 

25 

32 

BS1-04 

Small  ISO80 

12 

30 

B  SI-05 

Small  ISO80 

14 

24 

BSI-06 

Small  ISO80 

4 

58 

BSI-07 

Small  ISO80 

16 

45 

BSI-13 

Small  ISO80 

6 

24 

BSI-14 

Small  ISO80 

6 

18 

BSI-15 

Small  ISO80 

6 

13 

BSI-18 

37mm 

10 

33 

BSI-19 

37mm 

2 

16 

BSI-20 

37mm 

12 

43 

KSI-04 

Small  ISO80 

11 

47 

KSI-05 

Small  ISO80 

17 

40 

KSI-06 

Small  ISO80 

17 

41 

KSI-07 

Small  ISO80 

7 

14 

KSI-08 

Small  ISO80 

11 

45 

KSI-09 

Small  ISO80 

2 

13 
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The  average  seed  item  offset  for  the  TEMTADS  2x2  was  10.2cm,  whereas  the  seed  item  offsets 
for  the  EM6 1 -MK2  averaged  32cm.  EM6 1 -MK2  data  were  collected  using  a  2  ft.  transect  spacing, 
on  average  while  TEMTADS  2x2  array  data  yield  transect  spacing  of  40cm  or  less  from  sensor  to 
sensor.  These  results  suggest  that  the  improved  spatial  resolution  achieved  by  the  TEMTADS  2x2 
array  of  smaller  footprint  sensors  leads  to  higher  accuracy  in  detection  of  subsurface  items  of 
interest.  Figure  7-1  shows  two  examples  of  seed  items  as  detected  by  each  sensor.  The  TEMTADS 
2x2  location  is  shown  as  a  black  “x”,  whereas  the  EM61-MK2  detected  locations  are  shown  as  a 
blue  “x”. 

Figure  7-1.  Comparison  of  TEMTADS  2x2  and  EM61-MK2  Detection  Performance 


TEMTADS  2x2  Data 

EM61-MK2  Data 

© 

|©  - 

vy 

BSI-01 

BSI-01 

KSI-04 

KSI-04 

//  © 

© 

(Osl 

TEMTADS  2x2 Target  Location 

EM61 -MK2  Target  Location 

As  detailed  in  Table  7-3,  BSI-01  and  KSI-04  had  TEMTADS  2x2  detection  offsets  of  5cm  and 
11cm,  respectively.  These  same  seed  items  had  EM61-MK2  detection  offsets  of  47cm  in  both 
cases.  In  both  scenarios  the  EM61-MK2  data  showed  a  double  peak  over  the  seed  item,  with  the 
closest  peak  being  at  the  47cm  offset  distance.  TEMTADS  2x2  displayed  a  single  consolidated 
anomalous  response  in  both  cases.  This  is  of  importance,  as  each  of  the  reacquired  locations  in 
both  of  these  EM61-MK2  scenarios  (two  reacquired  locations  per  seed  item)  will  be  greater  than 
40cm  from  the  actual  source  location,  and  will  result  in  a  minimum  of  3  cued  field  measurements 
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to  adequately  characterize  the  source  (an  initial  measurement  at  each  reacquired  location,  with  at 
least  one  additional  recollect  from  one  of  the  locations  based  on  the  in-field  inversion  results). 
Conversely,  the  reacquired  locations  as  detected  by  the  TEMTADS  2x2  will  most  likely  only 
require  a  single  cued  measurement. 

7.5  OBJECTIVE:  CUED  INTERROGATION  OF  ANOMALIES 

This  objective  evaluated  the  positioning  of  the  instrument  during  data  collection  in  relation  to  the 
actual  anomaly  location.  The  metric  for  this  objective  was  considered  to  be  met  if  the  center  of  the 
instrument  was  positioned  within  40  cm  of  the  actual  anomaly  location  for  100%  of  the  cued 
anomalies. 

Comparison  of  TEMTADS  2x2  cued  locations  to  the  actual  location  of  recovered  items  is  not 
possible.  Detailed  intrusive  infonnation  and  locations  of  recovered  items  were  not  collected  by  the 
contractor  performing  the  MEC  removal  activities. 

As  a  result,  this  objective  was  evaluated  by  comparing  the  offset  between  the  center  of  the 
TEMTADS  2x2  array  and  the  fit  location  of  each  source.  Of  the  1,195  cued  measurements  that 
were  analyzed,  1,127  were  within  the  40-cm  offset  metric,  and  68  were  outside  of  40  cm.  39  of 
these  exceedances  were  from  the  977  targets  (4%  of  TEMTADS  targets)  selected  from  dynamic 
TEMTADS  2x2  data,  and  29  were  from  the  218  targets  (13%  of  the  EM61  targets)  selected  from 
EM61-MK2  data.  The  increased  exceedance  rate  at  targets  selected  from  the  lower  resolution 
EM61-MK2  dynamic  detection  survey  suggests  that  the  data  in  which  the  target  locations  are 
selected  from  will  influence  failure  rates.  This  objective  was  not  achieved  because  only  94%  of 
the  cued  measurements  were  within  the  40-cm  offset. 

7.6  CORRECTLY  CLASSIFY  QC  SEEDS  AND  CORRECTLY  CLASSIFY  NATIVE 
AND  POPULATION  SEED  ITEMS 

This  objective  evaluated  the  effectiveness  of  the  advanced  classification  process  to  properly 
classify  TOI  present  within  the  survey  area.  The  objective  was  considered  to  be  met  if  100%  of 
the  QC  seeds,  population  seeds,  and  native  TOI  were  placed  on  the  TOI  list. 

A  ranked  anomaly  list  was  submitted  to  the  ESTCP  Program  Office  for  evaluation.  This  objective 
was  met,  because  all  TOI  were  properly  classified  as  category  1  digs  (likely  TOI). 

7.7  OBJECTIVE:  CORRECTLY  IDENTIFY  GROUP 

This  objective  evaluated  the  effectiveness  of  the  advanced  classification  process  to  properly  assign 
each  excavated  TOI  and  non-TOI  into  the  small,  medium,  or  large  grouping.  The  objective  was 
considered  to  be  met  if  85%  of  the  anomalies  placed  on  the  dig  list  were  properly  grouped. 

Dig  results  for  the  ranked  anomaly  list  submitted  to  the  ESTCP  Program  Office  were  analyzed  to 
verify  size  groupings.  Intrusive  investigations  were  perfonned  to  the  specifications  of  the  3rd-party 
contractor’s  work  plan,  and  did  not  follow  the  standard  ESTCP  intrusive  investigation 
requirements,  so  dig  data  obtained  did  not  include  precise  locations,  depths,  photos  or  descriptions 
of  items  recovered.  Since  precise  data  were  not  available,  a  qualitative  analysis  was  performed  on 
the  dig  results  to  detennine  if  the  proper  group  was  assigned  to  each  item.  Of  the  220  anomalies 
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placed  on  the  dig  list,  186  were  assigned  to  the  correct  size  group,  3  were  assigned  an  incorrect 
group,  and  31  did  not  have  adequate  data  to  make  a  comparison.  This  objective  was  met,  as  98% 
of  the  anomalies  that  had  adequate  dig  data  for  a  qualitative  comparison  were  correctly  classified. 

7.8  OBJECTIVE:  CORRECT  ESTIMATION  OF  EXTRINSIC  TARGET 
PARAMETERS 

This  objective  evaluated  the  accuracy  of  the  target  parameters  that  are  estimated  during  the  data 
inversion  process  by  comparing  the  predicted  extrinsic  target  parameters  to  the  measured  results 
recorded  during  the  intrusive  investigation.  This  objective  was  considered  to  be  met  if  the 
estimated  X  and  Y  locations  were  within  15  cm  and  the  estimated  depths  were  within  10  cm. 

Detailed  locations  and  depths  of  items  recovered  were  not  recorded  during  the  intrusive 
investigation,  so  a  complete  evaluation  of  all  cued  data  could  not  be  performed.  Therefore,  this 
objective  was  evaluated  by  comparing  the  fit  locations  derived  from  the  cued  TEMTADS  data  to 
the  RTK  surveyed  locations  and  depths  of  the  seed  items  installed  within  the  TEMTADS 
demonstration  area.  Measured  offsets  for  the  X,  Y  and  Z  orientations,  as  well  as  the  standard 
deviation  of  the  offsets  for  each  orientation  are  detailed  in  Table  7-6. 

This  objective  was  met,  as  the  standard  deviation  for  each  of  the  X  and  Y  horizontal  offset  and  the 
Z  vertical  offset  were  <15  cm  and  <10  cm  respectively. 

7.9  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

This  objective  concerns  the  component  of  the  classification  problem  that  involves  false  alann 
reduction.  The  metric  for  this  objective  is  the  number  of  cued  anomalies  that  can  be  correctly 
classified  as  non-TOI.  The  objective  was  considered  to  be  met  if  more  than  50%  of  the  non-TOI 
items  were  correctly  labeled  as  non-TOI. 

Dig  results  for  the  initial  ranked  anomaly  list  submitted  to  the  ESTCP  Program  Office  were  used 
to  assess  the  number  of  non-TOI  that  were  correctly  classified.  Results  are  detailed  in  Table  7-6. 

This  objective  was  met,  because  81%  of  non-TOI  were  correctly  classified.  In  this  classification 
scenario,  100%  of  TOI  were  correctly  classified  while  achieving  a  false  positive  rate  of  only  18%. 
A  Receiver  Operating  Curve  (ROC)  generated  from  the  dig  results  of  the  submitted  ranked 
anomaly  list  is  shown  in  Figure  7-2. 
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Table  7-6.  Extrinsic  Target  Parameters 


Seed  ID 

Seed  Type 

X  Offset  (cm) 

Y  Offset  (cm) 

Z  Offset  (cm) 

B  SI-02 

Small  ISO80 

1.32 

3.40 

3.23 

B  SI-03 

Small  ISO80 

16.46 

3.45 

6.35 

BS1-04 

Small  ISO80 

5.96 

7.19 

4.50 

BSI-05 

Small  ISO80 

10.10 

0.95 

0.97 

BSI-07 

Small  ISO80 

3.70 

1.29 

0.74 

BSI-13 

Small  ISO80 

0.20 

3.63 

5.74 

BSI-14 

Small  ISO80 

0.97 

0.23 

1.64 

BSI-15 

Small  ISO80 

2.71 

7.66 

0.70 

BSI-16 

37mm 

2.86 

5.53 

1.73 

BSI-18 

37mm 

2.48 

1.14 

5.32 

BSI-19 

37mm 

6.07 

0.12 

6.32 

B  SI-20 

37mm 

9.18 

7.64 

12.57 

KSI-02 

Small  ISO80 

0.62 

1.35 

0.23 

KS1-04 

Small  ISO80 

3.46 

0.64 

1.29 

KSI-05 

Small  ISO80 

3.34 

1.67 

0.32 

KSI-06 

Small  ISO80 

0.60 

6.74 

0.17 

KSI-07 

Small  ISO80 

11.17 

26.88 

8.30 

KSI-08 

Small  ISO80 

9.00 

2.10 

1.13 

KSI-09 

Small  ISO80 

1.30 

2.37 

1.32 

KSI-10 

Small  ISO80 

1.81 

2.44 

2.03 

Standard  Deviation  (a) 

4.40 

5.87 

3.30 

Table  7-7.  Predicted  vs.  Actual  Classification  Results 


Categorized  TOI 

Categorized  Non-TOI 

Predicted 

243 

952 

Actual 

19 

1,176 

%  Correctly  Classified 

100% 

81% 
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Figure  7-2.  Guam  NRP  Area  Final  Receiver  Operating  Curve 


Guam  Weston  MultiCrit  None  2x2  Custom  s2  v2 


7.10  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

This  objective  evaluated  how  well  the  modeled  results  of  the  inversion  process  correlated  to  the 
observed  data.  A  fit  coherence  metric  is  calculated  for  each  model  during  data  inversion,  and  is 
used  as  the  basis  for  detennining  whether  reliable  parameters  could  be  estimated  from  the  data. 
The  objective  was  considered  to  be  met  if  reliable  parameters  could  be  estimated  for  >  95%  of  the 
anomalies  on  each  sensor  anomaly  list. 

Modeled  results  with  a  fit  coherence  of  less  than  0.8  were  placed  in  the  ‘cannot  analyze’  category. 
This  objective  was  met,  because  98%  of  the  cued  data  collected  inverted  with  a  fit  coherence 
greater  than  0.8. 
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8.  COST  BENEFIT  ANALYSIS 


The  cost  assessment  for  the  Andersen  AFB  demonstration  at  NRP  area  includes  a  summary  list  of 
the  project  costs  and  potential  savings  from  the  classification  process. 

8.1  COST  MODEL 

The  costs  for  the  NRP  area  field  demonstration  included  the  seeding,  TEMTADS  2x2  surveys,  and 
data  processing.  These  costs  are  summarized  in  Table  8-1. 


Table  8-1.  Details  of  Project  Costs 


Phase  of  Work 

Elements  of  Work 

Estimated  Costs 

Mobilization/  Demobilization 

Travel  for  a  geophysicist  and  UXO  technician 
and  shipment  of  TEMTADS  2x2  and  other 
equipment 

$16,000 

Site  Setup 

Site  prep,  seeding,  IVS  installation,  weather 
delays 

$15,000 

Dynamic  Detection  Survey 

Includes  effort  for  field  data  collection  (2.97 
acres),  equipment,  expenses,  labor  for 
geophysicists  and  UXO  technicians, 
processing/  anomaly  selection,  weather  delays/ 
downtime  and  local  subcontractor  support  (10 
days) 

$34,000 

Total  cost  per  acre  for  dynamic  survey 

$11,447/  acre 

Cued  Survey 

Equipment,  data  collection,  labor  for 
geophysicists  and  UXO  technicians  and 
expenses,  weather  delays/  downtime  and  local 
subcontractor  support  (5  days) 

$51,000 

Data  processing  and  classification 

$34,000 

Total  cost  per  anomaly  for  cued  survey  (1,195 
anomalies) 

$7 1/anomaly 

8.2  COST  DRIVERS 

This  analysis  presents  and  compares  the  costs  of  the  dynamic  and  cued  surveys  to  the  outcome  of 
the  classification  process.  Due  to  logistical  considerations  with  the  location  of  Guam,  travel 
(mobilization/  demobilization)  as  well  as  Site  Setup  has  been  separately  reported  in  Table  8-1. 
WESTON  also  procured  the  help  from  a  local  subcontractor  to  provide  additional  UXO  Technician 
escort  and  avoidance  support,  vehicles,  storage  and  equipment.  TEMTADS  2x2  shipping  was 
approximately  $12,000.  Expenses  and  some  labor  were  expended  during  weather  delays  and 
downtime  due  to  equipment  issues. 

A  total  of  2.97  acres  of  dynamic  detection  survey  data  was  collected  using  TEMTADS  2x2  over 
the  course  of  5  days.  The  total  cost  for  the  dynamic  detection  survey  was  approximately  $34,000 
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equating  to  $1 1,447  per  acre.  The  cued  survey,  data  analysis  and  classification  at  1,195  anomalies 
totaled  $85,000  equating  to  $71  per  anomaly. 

WESTON  considers  the  costs  incurred  for  both  the  dynamic  detection  and  cued  surveys  an 
overestimation  of  the  actual  costs  that  would  be  necessary  for  future  projects  because  these  costs 
included  significant  logistical  considerations  and  a  lengthy  setup  and  orientation  process.  An 
average  of  0.6  acres  per  day  was  achieved  for  dynamic  detection  surveys.  A  total  of  1.6  acres  per 
day  was  achieved  when  a  full  day  of  data  collection  was  possible.  An  average  of  120  anomalies 
per  day  was  achieved  during  cued  surveys.  A  total  of  244  anomalies  per  day  was  achieved  when  a 
full  day  of  data  collection  was  possible. 

8.3  COST  BENEFIT 

All  anomalies  were  intrusively  investigated  as  part  of  the  MEC  removal  action. 

The  classification  process  eliminated  81%  of  anomalies  that  would  be  required  to  be  intrusively 
investigated.  A  savings  of  $16,000  or  9%  of  the  total  cost  would  be  saved  based  on  the  following 
assumptions  and  factors: 

•  The  1,195  anomalies  spanned  4  acres. 

•  If  a  TEMTADS  2x2  dynamic  detection  survey  was  performed  across  the  entire  4  acres  of 
the  NRP  area  the  total  survey  cost  would  be  $45,788. 

•  The  total  cost  to  classify  all  anomalies  was  $85,000. 

•  We  assume  the  average  wraparound  cost  of  $150  per  anomaly  for  the  intrusive  work  the 
total  cost  for  1,195  anomalies  would  be  $180,000. 

•  The  classification  process  reduced  the  dig  list  to  243  anomalies  from  1,195  anomalies. 

•  At  $150  per  anomaly,  the  total  intrusive  work  would  be  $36,450  for  243  anomalies. 

•  By  using  geophysical  classification  process  under  these  assumptions,  the  total  MEC 
removal  action  cost  is  $167,238. 

•  Under  this  scenario,  a  savings  of  approximately  $12,762  would  be  realized  or  7%  of  the 
total  cost  without  using  geophysical  classification. 
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9.  IMPLEMENTATION  ISSUES 

The  primary  implementation  issue  observed  during  the  demonstration  at  Guam  was  data  collection 
delays  due  to  minor  inclement  weather.  The  TEMTADS  2x2  system  is  not  weather  ruggedized, 
and  as  a  result,  cannot  be  used  in  even  slightly  adverse  weather  conditions  (light  rain,  misting, 
etc...)  without  the  possibility  of  moisture  entering  the  sensor  housing  clamshell  or  computer 
backpack.  Due  to  this,  upwards  of  10  field  days  were  lost  during  the  mobilization  due  to 
precipitation  and  moisture  issues,  or  system  downtime  most  likely  caused  by  moisture  infiltrating 
system  components. 

Minor  setbacks  also  occurred  during  the  project,  such  as  software  malfunctions,  computer 
component  failures,  failed  sensors,  data  collection  delays  due  to  extreme  weather  events  (tropical 
storms,  etc...).  These  types  of  setbacks  can  be  typical  of  any  site  and  should  be  expected  when 
planning  field  operations. 

Based  on  the  data  collection  experience  at  Guam,  the  following  suggestions  were  made  for 
improvement  of  the  system  and  sensor  platform. 

•  Weatherproof  the  system  -  Limited  protection  of  cables  and  electronics  is  built  into  the 
TEMTADS  2x2  system.  The  clamshell  is  not  sealed  to  outside  moisture  and  dust.  Sensor 
failures,  likely  due  to  moisture  entering  the  clamshell,  resulted  in  several  days  of  downtime 
and  system  troubleshooting.  A  rubber  gasket  sealing  the  top  and  bottom  clamshell  pieces, 
as  well  as  the  cable  harness  entering  the  clamshell,  is  a  possible  solution. 

•  Wheels  -  Currently  the  wheels  are  held  on  by  zip  ties,  as  opposed  to  nylon  bolts  or  cotter 
pins.  These  zip  ties  routinely  failed  on  almost  a  daily  basis,  leading  to  downtime  to  reattach 
wheels  and  recollect  affected  lines. 

•  Other  modifications  -  Other  modifications  discussed  included  a  modification  of 
EM3Dacquire  to  allow  for  display  of  dynamic  data  collection  progress  (plot  tracks  on  map, 
similar  to  the  MetalMapper  EM3D  interface),  and  the  option  to  import  a  flag/target  list  into 
TEMDatalogger  to  avoid  operator  input  errors. 
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Appendix  A:  Points  of  Contact 


POINT  OF 
CONTACT 

ORGANIZATION 

PHONE 

FAX 

EMAIL 

ROLE  IN  PROJECT 

Dr.  Herb  Nelson 

ESTCP  Program  Office 

4800  Mark  Center  Drive, 

Suite  17D08 

Alexandria,  VA  22350-3605 

571-372-6400  (V) 

703-696-2114  (F) 

202-215-4844  (C) 
herb.nelson@nrl.navy.mil 

Program  Manager  for 
Munitions  Response 

Ryan 

Steigerwalt,  PG 

Weston  Solutions,  Inc. 

1400  Weston  Way 
Westchester,  PA,  19380 

267-258-2672  (V) 

Ryan.  Steigerwalt@W  estonSolutions.com 

Lead  PI 
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